Papers with ITI baseline
Multi-Attribute Steering of Language Models via Targeted Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for steering large language models fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity. |
| Approach: | They propose a steering framework for selective token-level intervention across multiple attributes that enforcing sparsity and orthogonality among vectors for different attributes. |
| Outcome: | The proposed framework outperforms existing ITI and parameter-efficient fine-tuning approaches across question answering tasks and generative tasks. |